System and method of object inspection using multispectral 3d laser scanning

ABSTRACT

The invention includes a system and method for obtaining high-resolution 3D images of objects. The system includes three cameras and three light sources that have different wavelengths (e.g. a red light source, a blue light source and a green light source). Each camera simultaneously captures a color image of the object. A processor separates each of the red light images, the blue light images and the green light images into separate monochrome images using each of the red light source, blue light source and green light source. The quality of the images are not subject to limited resolution of conventional RBG images. Because three different wavelengths of light are used, the surface can be accurately imaged, regardless of its characteristics (e.g. reflectivity and transparency). The system is well suited for industrial uses that require a high volume of objects, particularly those of mixed material, to be rapidly inspected for defects as small as a few microns.

TECHNICAL FIELD

The invention relates to a system and method of three-dimensional (3D)imaging, and more specifically, to a system and method of producinghigh-resolution images across a wide field of view using multiple lasersfor inspecting small objects such as electronic components.

BACKGROUND

Various applications require high-resolution imaging across a wide fieldof view (FOV). In high-precision manufacturing, objects are inspected toensure the absence of flaws or irregularities. 3D vision can beessential as the inspection involves examining small but criticalfeatures on each component in different planes. For example, AutomatedOptical Inspection (AOI) systems are often used to analyze and evaluateelectrical circuits, including flat panel displays, integrated circuits,chip carriers and printed circuit boards.

In AOI systems, a sensor such as a camera scans an object to confirm thepresence of components and/or the absence of quality defects. It is anon-contact test method and is often used in multiple stages of amanufacturing process. Modern AOI systems often use three-dimensional(3D) laser scanning.

3D laser scanning is a non-contact, non-destructive technology thatdigitally captures the shape of physical objects using a line of laserlight. 3D laser scanners create “point clouds” of data from the surfaceof an object. A digital representation of the size and shape of aphysical object can be captured and saved in a computer.

Conventional AOI machines use two-dimensional (2D) measurements tocreate a 3D image. Typically, two cameras are placed horizontally fromone another to obtain different views of an object. By comparing the twoimages, the relative depth information can be obtained in the form of adisparity map. However, this approach can be unsuited for someindustrial uses because it can be slow and impractical for high volumeprocesses. Other limitations include limited sensitivity and thedetection of erroneous or false positive defects. Alternative methodshave been developed in an effort toward improving 3D imaging technology.

Triangulation based 3D laser scanners shine a laser on a substrate anduse a camera to look for the location of the laser dot. Depending on howfar away the laser strikes a surface, the laser dot appears at differentplaces in the camera's field of view. This technique is calledtriangulation because the laser dot, the camera and the laser emittercan form a triangle. The length of one side of the triangle, thedistance between the camera and the laser emitter are known. The angleof the laser emitter corner is also known. The angle of the cameracorner can be determined by the location of the laser dot in thecamera's field of view. In most cases a laser strip, instead of a singlelaser dot, is swept across the object to speed up the process. Whiletriangulation offers benefits to other techniques, alternativeapproaches often focus on overcoming its limitations.

For example, U.S. Pat. No. 5,406,372 describes a quad flat pack (QFP)chip package inspection system and method. Its purpose is to inspectleads on QFP chip packages. A package is scanned using two lasers and asensor for detecting and processing reflected laser light from theobject. Reflected light from each of the lasers provides data from eachlead on the two transverse parallel edges.

Similarly, WO2018-072433 A1 describes a three-dimensional scanningmethod that uses multiple lasers. A 3D object is scanned using multiplelasers of differing wavelengths and multiple cameras for detecting andprocessing reflected laser emitted from an object. Laser contour linesprojected onto a surface of a scanned object are photographed by twocameras. A spatial three-dimensional point cloud data can be constrictedaccording to the trigonometry principle and the epipolar constraintprinciple.

U.S. Pat. No. 6,750,974 describes a method of scanning an object (e.g. awafer) using a laser and camera for triangulation based sensordetection. The system aims to detect both specular and diffusereflections of the object. As with other systems, a detector collectsdata from an illuminated surface. A second detector collects data from asecond location that is fused with data from the first detector.However, as with other approaches, this system has shortcomings.

Conventional methods of laser scanning can be ineffective withsubstrates that are irregularly shaped or composed of more than onematerial. It can be difficult to identify portions of the substrate thatare occluded or lie on an axis parallel to the source of laser light.Further, 3D laser scanning using a single laser on mix material mayprovide inadequate or poor scan data. The intensity and wavelength oflaser light can be ineffective for particular substances. Using multiplescans with different laser parameters can mitigate some of the aboveissues but increases the time needed to complete the scanning.Sequential scans can also introduce errors due to mechanical offsetsbetween scans. An improved system and method should overcome theseissues. It should have a fast inspection speed including quick catches,high accuracy of inspection accuracy and high consistency of inspection.

A need, therefore, exists for a system and method to overcome theshortcomings of conventional 3D imaging systems. Specifically, there isa need for an improved system and method for obtaining 3D images of thesurfaces of objects. The system should be capable of detecting flaws orirregularities on substrates that are composed of different materials.It should also be capable of operating at a high speed so that manyobjects can be inspected or screened in a short period of time.

SUMMARY OF THE INVENTION

The following summary is provided to facilitate an understanding of someof the innovative features unique to the disclosed embodiment and is notintended to be a full description. A full appreciation of the variousaspects of the embodiments disclosed herein can be gained by taking intoconsideration the entire specification, claims, drawings, and abstractas a whole.

Embodiments of the invention include a system for obtaining ahigh-resolution 3D image of an object. The system can include (a) afirst camera at a first location, (b) a second camera at a secondlocation, (c) a third camera at a third location, (d) a red laser lightsource, (e) a blue laser light source, (f) a green laser light sourceand (g) a processing unit. The first camera can capture a first colorimage of the object and the processor can separate the first color imageinto a first monochrome image using the red laser light source, a secondmonochrome image using the blue laser light source and a thirdmonochrome image using the green laser light source. The second cameracan capture a second color image of the object and the processor canseparate the second color image into a fourth monochrome image using thered laser light source, a fifth monochrome image using the blue laserlight source and a sixth monochrome image using the green laser lightsource. Similarly, the third camera can capture a third color image ofthe object and the processor can separate the third color image into aseventh monochrome image using the red laser light source, an eighthmonochrome image using the blue laser source and a ninth monochromeimage using the green laser light source. The processing unit cancombine the first monochrome image, the second monochrome image, thethird monochrome image, the fourth monochrome image, the fifthmonochrome image, the sixth monochrome image, the seventh monochromeimage, the eighth monochrome image and the ninth monochrome image into asingle high-resolution 3D image of the object.

The red laser light source, the blue laser light source and the greenlaser light source can be aligned at different points respective to eachother on a y-axis and/or a z-axis. The first camera, the second cameraand the third camera can be aligned at different points respective toeach other on a y-axis and/or a z-axis. The processor can generate afirst depth map, a second depth map, a third depth map, a fourth depthmap, a fifth depth map, a sixth depth map, a seventh depth map, aneighth depth map and a ninth depth map based on point cloud data Theprocessor can also generate a map of nine rows in nine depth using thefirst monochrome image, the second monochrome image, the thirdmonochrome image, the fourth monochrome image, the fifth monochromeimage, the sixth monochrome image, the monochrome seventh image, theeighth monochrome image and the ninth monochrome image by triangulation.

Embodiments also include a method of obtaining a high-resolution 3Dimage of an object, comprising steps of (a) moving the object toward theintersection of a laser fan plane (b) illuminating the object with a redlaser light source, (c) illuminating the object with a blue laser lightsource, (d) illuminating the object with a green laser light source, (e)capturing a first color image of the object with a first camera, (f)simultaneously capturing a second color image of the object with asecond camera, (g) simultaneously capturing a third color image of theobject with a third camera, (h) separating the first color image into afirst monochrome image using the red laser light source, a secondmonochrome image using the blue laser light source and a thirdmonochrome image using the green laser light source, (i) separating thesecond color image into a fourth monochrome image using the red laserlight source, a fifth monochrome image using the blue laser light sourceand a sixth monochrome image using the green laser light source, (j)separating the third color image into a seventh monochrome image usingthe red laser light source, an eighth monochrome image using the bluelaser source and a ninth monochrome image using the green laser lightsource, (k) converting the first monochrome image, the second monochromeimage, the third monochrome image, the fourth monochrome image, thefifth monochrome image, the sixth monochrome image, the seventhmonochrome image, the eighth monochrome image and the ninth monochromeimage into point-cloud data; and (l) creating a single high-resolution3D image from the point-cloud data.

The method can include a step of moving the object to capture additionalcolor images to generate additional point cloud data. It can alsoinclude a step of identifying and/or filtering erroneous data points asoutlying point cloud data and a step of creating and analysing a depthmap image for defects on the object. Further, the method can include astep of adjusting the region of interest (ROI) the first camera, thesecond camera and/or the third camera to capture different surfaceheights on the object.

INTRODUCTION

A first aspect of the invention is a system for obtaining 3D images ofan object for inspection and/or identification of surface defects thatuses multiple laser light sources and triangulation to improve theaccuracy of point cloud data.

A second aspect of the invention is a system that uses multiple lasersand detectors that are aligned apart from one another on two axis (e.g.x and z) to scan the surface of an object.

A third aspect of the invention is a method of separating a color imageinto monochrome images and creating a single high-resolution image frompoint cloud data.

A fourth aspect of the invention is a system for obtaining a 3D image ofan object of mixed material wherein a single scanned image is capturedusing light sources of different wavelengths.

A fifth aspect of the invention is a system for inspecting an object ofmixed material using light sources of different wavelengths and multiplesensors/detectors.

A sixth aspect of the invention is a method of inspecting an object ofmixed material using light sources that are aligned apart from oneanother on two axis (e.g. x and z).

A seventh aspect of the invention is a method of calibrating a systemwith multiple laser light sources using a jig or block of knowndimensions.

Definitions

The term “albedo” refers to a measure of the diffuse reflection of solarradiation out of the total solar radiation received by an astronomicalbody. It is dimensionless and measured on a scale from 0 to 1. Surfacealbedo refers to the ratio of irradiance reflected to the irradiancereceived by a surface.

The term “bayer filter” refers to a color filter array (CFA) forarranging RGB color filters on a square grid of photosensors. Itsparticular arrangement of color filters is common in single-chip digitalimage sensors used in digital cameras, camcorders, and scanners tocreate a color image. The filter pattern is 50% green, 25% red and 25%blue, hence is also referred to as BGGR, RGBG, GRGB or RGGB.

The term “bilateral filter” refers to a non-linear, edge-preserving, andnoise-reducing smoothing filter for images. It replaces the intensity ofeach pixel with a weighted average of intensity values from nearbypixels. This weight can be based on a Gaussian distribution. Crucially,the weights depend not only on Euclidean distance of pixels, but also onthe radiometric differences (e.g., range differences, such as colorintensity, depth distance, etc.). This preserves sharp edges.

The term “centroid” or “geometric center of a plane figure” refers tothe arithmetic mean position of all the points in the figure.Informally, it is the point at which a cutout of the shape could beperfectly balanced on the tip of a pin.

The term “depth map” refers to an image or image channel that containsinformation relating to the distance of the surfaces of scene objectsfrom a viewpoint. In 3D computer graphics, a depth map is an image orimage channel that contains information relating to the distance of thesurfaces of scene objects from a viewpoint. The term is related to andmay be analogous to depth buffer, Z-buffer, Z-buffering and Z-depth.

The term “diffuse reflection” refers to the reflection of light or otherwaves or particles from a surface such that a ray incident on thesurface is scattered at many angles rather than at just one angle as inthe case of specular reflection. An ideal diffuse reflecting surface issaid to exhibit Lambertian reflection, meaning that there is equalluminance when viewed from all directions lying in the half-spaceadjacent to the surface.

The term “epipolar geometry” refers to the geometry of stereo vision.When two cameras view a 3D scene from two distinct positions, there area number of geometric relations between the 3D points and theirprojections onto the 2D images that lead to constraints between theimage points.

The term “integrated circuit,” “IC” or “wafer” refers to a small complexof electronic components and their connections that is produced in or ona small slice of material such as silicon.

The term “laser fan plane” refers to a two-dimensional surface withindeterminate width and length, zero thickness and zero curvature castedby a line laser with a fan angle. Laser light will form a fan-shape thatresults from laser divergence. Laser beams diverge to a certain degree.The beam divergence (i.e. laser fan plane) defines how much the beamspreads out over increasing distance from the optical aperture.

The term “optical axis” refers to a line along which there is somedegree of rotational symmetry in an optical system such as a camera lensor microscope. The optical axis is an imaginary line that defines thepath along which light propagates through the system, up to firstapproximation.

The term “point cloud” refers to a large set of data points in space.Laser scanners can collect point cloud data from surface points on anobject. The data can be used for quality inspection and/or visualizationof the object.

The term “Quad Flat Package” or “QFP” refers to a surface mountintegrated circuit package with “gull wing” leads extending from each ofthe four sides.

The term “semiconductor chip,” as used herein, refers to an integratedcircuit or monolithic integrated circuit (also referred to as an IC, achip, or a microchip) which is a set of electronic circuits on one smallplate (“chip”) of semiconductor material, normally silicon.

The term “specular reflection” or “regular reflection” refers to themirror-like reflection of waves from a surface. Each incident ray isreflected at the same angle to the surface normal as the incident ray,but on the opposing side of the surface normal in the plane formed byincident and reflected rays.

The term “Thin Small Outline Package” or “TSOP” refers to a type ofsurface mount Integrated Circuit (IC) package. They typically have leadson two sides and are often used for RAM or Flash memory ICs due to theirhigh pin count and small volume.

The term “triangulation” refers to a process of determining a point in3D space given its projections onto two or more images. Its location canbe determined if the parameters of the camera projection function from3D to 2D for the cameras are known. For example, an incident laser beamstrikes an object. The instantaneous field of view of the sensor canthereafter be determined by a sensor and optical system. Height and grayscale information are provided from a signal processor to a computer forimage processing and analysis.

The term “voxel” refers to a value on a regular grid inthree-dimensional space. As with pixels in a bitmap, voxels themselvesdo not typically have their position (their coordinates) explicitlyencoded along with their values. Instead, rendering systems infer theposition of a voxel based upon its position relative to other voxels(i.e., its position in the data structure that makes up a singlevolumetric image). In contrast to pixels and voxels, points and polygonsare often explicitly represented by the coordinates of their vertices.

BRIEF DESCRIPTION OF THE FIGURES

The drawings described herein are for illustrative purposes only ofselected embodiments and not all possible implementations, and are notintended to limit the scope of the disclosure.

FIG. 1A depicts an arrangement of components of a system for obtaining3D images of objects to inspect and/or detect small irregularities orflaws, according to one aspect of the invention.

FIG. 1B is a table listing the preferred laser parameters and power forproducing high-resolution images of substrates with various types ofsurfaces.

FIG. 2A depicts an arrangement of components of a multispectral systemfor obtaining high-resolution 3D images of small irregularities or flawson an object, according to one aspect of the invention.

FIG. 2B depicts an arrangement of components of a multispectral systemfor obtaining high-resolution 3D images of objects to inspect and/ordetect small irregularities or flaws, according to one aspect of theinvention.

FIG. 2C depicts a bottom view of an arrangement of components of amultispectral system for obtaining high-resolution 3D images of objectsfor inspection and/or detection of small irregularities or flaws,according to one aspect of the invention.

FIG. 3A is a flow chart that describes a process for obtaininghigh-resolution 3D images of an object using multiple cameras andmultiple light sources, according to one aspect of the invention.

FIG. 3B is a flow chart that describes the steps in processing data toobtain a high-resolution 3D image, according to one aspect of theinvention.

FIG. 4A is a flow chart that describes a process for obtaininghigh-resolution 3D images of objects using two cameras and two lightsources, according to one aspect of the invention.

FIG. 4B is a flow chart that describes the steps in processing data toobtain high-resolution 3D images, according to one aspect of theinvention.

FIG. 5A is a flow chart that describes the steps in a process forobtaining high-resolution 3D images of an object using three cameras andthree light sources, according to one aspect of the invention.

FIG. 5B is a flow chart that describes the steps of processing data toobtain high-resolution 3D images, according to one aspect of theinvention.

FIG. 6A is a perspective view of a calibration block, according to oneaspect of the invention.

FIG. 6B is also a perspective view of a calibration block, according toone aspect of the invention.

DETAILED DESCRIPTION OF THE INVENTION

An objective of the invention is to provide high-resolution images toinspect three-dimensional features on small objects with greater speedwith higher sensitivity than conventional approaches. Further, by usingdifferent wavelengths of light, different kinds of materials can bevisualized and inspected.

In conventional lens-based imaging systems, a large space-bandwidthproduct requires using higher magnification and larger lenses. The imagesensors are also made larger with higher pixel counts. In thealternative, image sensors with a smaller pixel pitch can be used whilemaintaining a large active area. However, larger components can beexpensive and make the system bulky and cumbersome. Moreover, suchsystems can be slow and impractical for industrial uses.

Another objective of the invention is to improve the accuracy of “pointcloud” data generated of the contours and geometry of an object. Theinvention utilizes the premise of laser triangulation between a laserdot or line on the object, a camera and the laser emitted. However, thequality of the scanned data using this triangulation is affected by theintensity of the laser (density of the point cloud) and the propertiesof the object surface to be scanned. Because many objects are a mix ofmaterials and properties with either a spectral reflection or diffusereflection, the laser scanning parameters must be optimized in order toimprove the quality and accuracy of the image.

A single light source can be used to generate a single image of anobject. Different light sources can be used to take multiple imagesconsecutively. The images can then be combined. An alternative approachcan use different light sources that are combined with mirrors and/orlenses. However, these methods can be tedious and lack precision.Regardless of the approach, the quality of scanned data is dependent onthe density of point cloud which can be affected by the wavelength andintensity of the laser as well as the surface of the object.

FIG. 1A depicts a system for recording an image and/or inspecting anobject such as an electronic component. The horizontal (x) axis,vertical (y) axis and z-axis are also depicted. An object 105 is placedin a field of view. A source of laser light 125 emits light onto theobject. Reflected light is detected with a sensor 120 such as a camera.Depending on how far away the laser strikes a surface, the laser dotappears at different places in the camera's field of view. The angle ofthe laser emitter corner is known and the angle of the camera corner canbe determined by looking at the location of the laser dot in thecamera's field of view. The shape and size of the triangle can bedetermined along with the location of the laser dot corner of thetriangle.

This process can be repeated to generate a series of data points (i.e. apoint cloud). An additional light source and/or sensor can be added tothe system to produce additional data points for the point cloud.

While effective for some applications, this approach has limitationsincluding inadequate quality of scanned data. For example, sections ofthe object on the opposite side of the camera can be occluded. An imageof these sections can be grainy and/or have poor resolution. The surfacetype can also affect the image. An object with high reflectivity may notlead to a clear image.

Quality of Scanned Data

A point cloud can be used to construct a 3D image. A sufficient amountof scanned data is needed for a point cloud of sufficient density. Thequality of scanned data is affected by the intensity of the laser linethat is projected onto an object. The intensity of the laser line alsodepends on the character of the surface of the object. The reflectivity,transparency, roughness, material and color can affect how the laserlight is detected by the camera.

Point Cloud Filtering

Point sets obtained by imaging devices such as 3D scanners are oftencorrupted with noise. Noise can be caused by tangential acquisitiondirection, changing environmental lights and/or a reflective objectmaterial. In one embodiment, each capture laser line is processed into arow of point clouds. The points overlap with points generated from otherlines.

Noise can be filtered by subsampling the points and converting the pointcloud into a representation of a piecewise continuous surface. In afirst method, using a single layer voxel grid, the z centroid of allpoints in each voxel is calculated. The x and y will be represented bythe center of the voxel. In a second method, a single layer voxel gridis used. A bilateral filter is applied to z values. The x and y will berepresented by the center of the voxel.

Surface Type

Given that reflectance is a directional property, most surfaces can bedivided into those that give specular reflection and those that givediffuse reflection. Specular reflection, also known as regularreflection, is the mirror-like reflection of waves, such as light, froma surface. In this process, each incident ray is reflected at the sameangle to the surface normal as the incident ray, but on the opposingside of the surface normal in the plane. For specular surfaces, such asglass or polished metal, reflectance will be nearly zero at all anglesexcept at the appropriate reflected angle. Reflected radiation willfollow a different path from incident radiation for all cases other thanradiation normal to the surface.

Diffuse reflection is the reflection of light or other waves orparticles from a surface such that a ray incident on the surface isscattered at many angles rather than at just one angle as in the case ofspecular reflection. An ideal diffuse reflecting surface is said toexhibit Lambertian reflection, meaning that there is equal luminancewhen viewed from all directions lying in the half-space adjacent to thesurface. For diffuse surfaces, such as matte white paint, reflectance isuniform. Radiation is reflected in all angles equally or near-equally.Most real objects have some mixture of diffuse and specular reflectiveproperties.

Optimization

Depending on the surface characteristic of a substrate, the laserparameters can be adjusted to improve the quality of the image. Forexample a low power laser is better suited for use with substrate withhigh reflectivity. FIG. 1B depicts the preferred laser power andparameters for various surface types. However, a substrate can becomposed of different materials, each with different characteristics.For example, a Quad Flat Package (QFP) typically has both plastic andmetal surfaces. The plastic surface will exhibit diffuse reflectionwhile the metal surface will exhibit specular reflection. Similarly, aBall Grid Array (BGA) usually has both metal and plastic components.

3D Laser scanning using a single laser on a substrate of mixed materialcan lead to inadequate or poor quality data. Using multiple scans withdifferent laser parameters can mitigate some of the above issues butincreases the time needed to complete the scanning. Moreover, scanningat different times introduces errors due to mechanical offset betweenscans.

In the present invention, an object is scanned using multiplewavelengths of light simultaneously. FIG. 2A depicts a system forimaging small objects such as electronic components using multiplesources of laser light and sensors. An object 105 can move across afield of view. A first source of laser light 125 emits light onto theobject. Simultaneously, a second source of laser light 115 emits lightonto the object. Reflected light is detected with a first sensor 120(e.g. camera) and a second sensor 110. The first and second sensor arepreferably situated apart from one another on the z-plane.

Each sensor (110, 120) can detect light from the first source of laserlight 125 and the second source of laser light 115. The source of laserlight can be distinguished by its wavelength. Each sensor will capturetwo lines that are projected across the object. These lines can providetwo sets of point clouds calculated which improves speed. Each locationcan scan two times from two lasers from different angles. However,specular reflection will occur from certain angles as well as occlusion.

Similarly, FIG. 2B depicts a system for imaging small objects such aselectronic components. An object 105 can move across a field of view. Afirst source of laser light 125 emits light onto the object.Simultaneously, a second source of laser light 115 emits light onto theobject along with a third source of laser light 135. Each source uses adifferent wavelength of laser light. Reflected light is detected with afirst sensor 120, a second sensor 110 and a third sensor 140. The first,second and third sensor are preferably situated apart from one anotheron the z-plane.

FIG. 2C depicts the arrangement of the first, second and third sourcesof laser light from a different perspective. The laser lights arepreferably situated apart from one another on the y-plane and thez-plane. With this arrangement, areas of the object 105 are not subjectto occlusion.

Each camera will capture three lines that are projected across theobject. These lines can provide three sets of point clouds whichimproves speed. Each location can scan three times from three lasersfrom different angles. Specular will occur from only a narrow angle,therefore only one out of three points may encounter specularreflection. With laser projecting from three angles, occlusion will bereduced. Further, any outlier points can be filtered using software.

FIG. 3A is a flowchart 100 that lists the steps in a preferred method ofcreating a high-resolution 3D image of an object for inspection. First,the object is placed in the field of view of the cameras 105. A movingobject can also be imaged, in which case, the other components areactivated at when the substrate is within the field of view. The nextseries of steps occur simultaneously. The object is illuminated with ared laser light 110 and the image is recorded 115. The object isilluminated with a green laser light 120 and the image is recorded 125.The object is illuminated with a red laser light 130 and the image isrecorded 135. The data is compiled from the three images.

The red, green and blue laser line cast over the subject can be compiledinto a point cloud 140. The point cloud is used to create an image ofthe substrate. FIG. 3B is a flowchart that lists the steps in utilizingthe data 200. The data is filtered based on intensity over bloom points150. Next, the point cloud data is merged together to form the surfaceof the subject 160. Any outlier points due to specular reflection can befiltered 170.

The effective camera sensor width and height, also known as the regionof interest (ROI), can be reduced to improve the image data read outspeed. Under normal setup, the ROI of all cameras capture the sameregion of a target surface. This is preferred when the object is flat orhas slight differences in height. With an object that has a surface withsubstantial height variations, a different camera ROI can be used byeach camera to capture the different surface heights. The data can bemerged to form a final 3D map without a reduction in scanning speedand/or accuracy. Additionally, the two cameras can have an overlappingROI on the surface and merge data together with an object that has awarped surface.

When the surface material has a substantial variation in reflectivenessor albedo, the light intensity of the different lasers can be varied.One can be set to a high power and one to a low power. The low poweredlaser line will record the high reflective surface and the high powerlaser will record the low reflective region. The data can be merged ontoa 3D map to increase the dynamic range of the surface reflectivity anddetermine the albedo.

Dual Laser/Camera Approach

FIG. 4A is flowchart 95 that lists the steps in creating ahigh-resolution 3D image in detail. The process begins when an object(i.e. subject) is picked and placed in a field of view (FOV) 305. Inthis example, two laser light sources are used. Green laser light andred laser light are projected onto the object 310. The object can bemoved along a linear trajectory perpendicular to the laser line 315.Next, images of the object are recorded simultaneously with two cameras320. The imaging step can be repeated 325 to gather additional imagedata.

The next steps involve processing the images recorded from the twocameras. The following steps (330, 335, 340 and 345) involve collectingdata for a point cloud. Each involves determining the position of pointsprojected from laser light (red or green) and recorded by the first orsecond camera. Thereafter, the epipolar constraint is determined togenerate points from the red laser profile 350 and the green laserprofile 355. In the next step, an uncertainty analysis can be performedto filter the data base on intensity and/or filter over bloom points360.

FIG. 4B is a flowchart 96 that describes some of the steps in dataanalysis. After uncertainty analysis 360, data points can be placed on athree-dimensional (3D) map 365. The subject can be moved by a fixed step370. The process can end 375 upon creation of the 3D map (375, 380).Alternatively, the process can be repeated 380 as additional images/dataare collected.

System Calibration

Calibration is important to ensure the precision and reliability of thesystem. As object inspection often entails detecting minor deformations,the system must be calibrated to a high degree. Calibration of the x, yand z axis is necessary.

The system can be calibrated using a calibration block of knowndimensions. FIGS. 6A and 6B depict a perspective view of a block. Theblock is preferably square or rectangular in shape with symmetricalfeatures to assist in calibration. The top portion can include twoprotruded portions, the precise dimensions of which are known. Further,the distance between the protruded portions is known along with theangles of each side portion.

FIG. 6A depicts a calibration block 300. The block can be describedbased on five regions as labelled (1-5). The step (i.e. protruded)portions are represented by regions 2 and 4. As the distance betweenthese regions is known (a), it can be used to calibrate one or morelasers of the system. Similarly, the height of each of stepped region(β) can also be used to calibrate one or more lasers of the system.

Similarly, FIG. 6B depicts a calibration block 300. The plane of eachside of the steps can be extrapolated to a point of intersection. Thispoint can be determined for step 2 and step 4. The distance between thepoints (8) can also be used to calibrate one or more lasers of thesystem. Each laser can be calibrated independently using the block.

Working Example 3D Imaging and Inspection of a Ball Grid Array

A Ball Grid Array (BGA) usually has both metal and plastic components.Multispectral 3D laser scanning system that scans multiple wave-lengthscan be used for such substrates. In this example, the first source oflaser light and the second source of laser light emit differentwavelengths of light. One laser is configures so that its wavelength isoptimized for a reflective surface. Another laser is configures so thatits wave length is optimized for diffuse reflection. Either a dual orsingle camera is used to scan with both lasers simultaneously emittinglight.

For example, the first and second sources of laser light can be 450 nmand 650 nm respectively. This allows an object to be inspectedsimultaneously using multiple wavelengths of light. One wavelength oflight can be optimal for reflective surface. The other wavelength oflight can be optimal for diffuse reflection. Conventional approachestypically involve conducting multiple scans of an object which haslimitations. The use of two lasers can also overcomes a shadow that canbe present when a single light source is used.

A multispectral 3D laser scanning setup can use dual wave-length lightsources (e.g. red and green). One wavelength is optimized for reflectivesurfaces. The other wavelength is optimized for diffuse reflection.Either single or dual cameras scan in the presence of both lightsources. A third laser light source can be added to the system.

The system allows a scan of a substrate with multiple wavelengths at thesame instance (rather than multiple scans). This leads to greatersensitivity without the need to combine data from different time points.The system is well suited to variations of reflectivity on the surfaceof a substrate. It also allows spectral scanning as different materialscan react differently to different wavelengths of light.

The high-resolution image can be created by picking and placing anobject (e.g. electrical component) in a FOV, toward the intersection ofa laser fan plane. Next, the FOV is illuminated with a red laser lightsource, a blue laser light source and a green laser light source. Inthis example, three image capturing devices (i.e. color cameras) areused to simultaneously capture a first color image, a second color imageand a third color image of the object.

Next, each color image is separated into three monochrome images. Themonochrome images each utilize one wavelength (i.e. color) of laserlight (red, blue and green). Accordingly, nine images in total arecreated at this step. Analysis of the images entails converting themonochrome images into a row of point-cloud data. Erroneous data pointscan be identified as outlying points and deleted. A first, second,third, fourth, fifth, sixth, seventh, eighth and a ninth depth map canbe created using the point cloud data. Thereafter, a singlehigh-resolution image can be created from the point-cloud data. Thehigh-resolution image can be analyzed to identify imperfections and/ordefects on the object.

The system can include a computer that contains software loaded thereonor executable by the computer to process raw images that are output fromthe sensor output of the image sensor. Images can be transferred to thecomputer using a conventional cable or wirelessly transmitted to thecomputer. The software can include the reconstruction algorithmdescribed herein which may be embodied in MATLAB® or other program(e.g., C language). The software is executed by one or more processorscontained in the computer. To improve performance, the computer caninclude a processor that is part of a graphics processing unit (GPU) tospeed the image reconstruction process. The computer can include apersonal computer, server, laptop or the like. The computer can also becoupled to or otherwise contain a monitor and one or more peripheraldevices (e.g., mouse, keyboard, or the like). Reconstructed images canbe displayed to the user on the monitor.

What is claimed is:
 1. A system for obtaining a high-resolution 3D imageof an object, comprising: a) a first camera at a first location; b) asecond camera at a second location; c) a third camera at a thirdlocation; d) a red laser light source; e) a blue laser light source; f)a green laser light source; and g) a processing unit, wherein the firstcamera captures a first color image of the object and the processorseparates the first color image into a first monochrome image using thered laser light source, a second monochrome image using the blue laserlight source and a third monochrome image using the green laser lightsource, wherein the second camera captures a second color image of theobject and the processor separates the second color image into a fourthmonochrome image using the red laser light source, a fifth monochromeimage using the blue laser light source and a sixth monochrome imageusing the green laser light source, wherein the third camera captures athird color image of the object and the processor separates the thirdcolor image into a seventh monochrome image using the red laser lightsource, an eighth monochrome image using the blue laser source and aninth monochrome image using the green laser light source, and whereinthe processing unit combines the first monochrome image, the secondmonochrome image, the third monochrome image, the fourth monochromeimage, the fifth monochrome image, the sixth monochrome image, theseventh monochrome image, the eighth monochrome image and the ninthmonochrome image into a single high-resolution 3D image of the object.2. The system of claim 1, wherein the red laser light source, the bluelaser light source and the green laser light source are aligned atdifferent points respective to each other on a y-axis and a z-axis. 3.The system of claim 1, wherein the processor generates a map of ninerows in nine depth using the first monochrome image, the secondmonochrome image, the third monochrome image, the fourth monochromeimage, the fifth monochrome image, the sixth monochrome image, themonochrome seventh image, the eighth monochrome image and the ninthmonochrome image by triangulation.
 4. The system of claim 1, wherein theprocessor generates a first depth map, a second depth map, a third depthmap, a fourth depth map, a fifth depth map, a sixth depth map, a seventhdepth map, an eighth depth map and a ninth depth map based on pointcloud data.
 5. The system of claim 1, wherein the first camera, thesecond camera and the third camera are aligned at different pointsrespective to each other on a y-axis and a z-axis.
 6. The system ofclaim 4, wherein the first depth map, the fourth depth map and theseventh depth map are aligned with one another on the x-axis; whereinthe second depth map, the fifth depth map and eighth depth map arealigned with one another on the x-axis; and wherein the third depth map,the sixth depth map and the ninth depth map are aligned with one anotheron the x-axis
 7. The system of claim 1, wherein the surface of theobject is comprised of more than one material, each material havingdifferent light reflection characteristics.
 8. The system of claim 1,wherein the object is a wafer or an integrated circuit (IC) package. 9.The system of claim 1, wherein a calibration jig with known dimensionsis used to position and/or calibrate the location of the red laser lightsource, the green laser light source and the blue laser light source.10. A method of obtaining a high-resolution 3D image of an object,comprising steps of: a) moving the object toward the intersection of alaser fan plane; b) illuminating the object with a red laser lightsource; c) illuminating the object with a blue laser light source; d)illuminating the object with a green laser light source; e) capturing afirst color image of the object with a first camera; f) simultaneouslycapturing a second color image of the object with a second camera; g)simultaneously capturing a third color image of the object with a thirdcamera; h) separating the first color image into a first monochromeimage using the red laser light source, a second monochrome image usingthe blue laser light source and a third monochrome image using the greenlaser light source; i) separating the second color image into a fourthmonochrome image using the red laser light source, a fifth monochromeimage using the blue laser light source and a sixth monochrome imageusing the green laser light source, j) separating the third color imageinto a seventh monochrome image using the red laser light source, aneighth monochrome image using the blue laser source and a ninthmonochrome image using the green laser light source; k) converting thefirst monochrome image, the second monochrome image, the thirdmonochrome image, the fourth monochrome image, the fifth monochromeimage, the sixth monochrome image, the seventh monochrome image, theeighth monochrome image and the ninth monochrome image into point-clouddata; and l) creating a single high-resolution 3D image from thepoint-cloud data.
 11. The method of claim 10, including a step of movingthe object to capture additional color images and generate additionalpoint cloud data.
 12. The method of claim 10, including a step ofidentifying and/or filtering erroneous data points as outlying pointcloud data.
 13. The method of claim 10, including a step of creating andanalysing a depth map image for defects on the object.
 14. The method ofclaim 10, wherein the red laser, the blue laser and the green laser arealigned at different points respective to each other on a y-axis and az-axis.
 15. The method of claim 10, wherein the surface of the object iscomprised of more than on material, each material having different lightreflection characteristics.
 16. The method of claim 10 including a stepof identifying defects, irregularities and/or surface flaws on theobject.
 17. The method of claim 10, including a step of adjusting theregion of interest (ROI) of at least one of the first camera, the secondcamera and the third camera to capture different surface heights on theobject.
 18. The method of claim 10, including a step of calibrating thered laser, the blue laser and the green laser with a calibration jig.